A Bounded Divergence Measure Based on The Bhattacharyya Coefficient

We introduce a new divergence measure, the bounded Bhattacharyya distance (BBD), for quantifying the dissimilarity between probability distributions. BBD is based on the Bhattacharyya coefficient (fidelity) , and is symmetric, positive semi-definite, and bounded. Unlike the Kullback-Leibler divergence, BBD does not require probability density functions to be absolutely continuous with respect to each other. We show that BBD belongs to the class of Csiszar f-divergence and derive certain relationships between BBD and well known measures such as Bhattacharyya, Hellinger and Jensen-Shannon divergence. Bounds on the Bayesian error probability are established with BBD measure. We show that the curvature of BBD in the parameter space of families of distributions is proportional to the Fisher information. For distributions with vector valued parameters, the curvature matrix can be used to obtain the Rao geodesic distance between them. We also discuss a potential application of probability distance measures in model selection.